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Using iReady AP2 Results to Make ESOL Exit Decisions 


Results at a Glance 
This Research Brief describes a model developed as an alternative method for making ESOL exit 
decisions in 2019-2020. Given the cancellation of the 2019-2020 Statewide student assessment, FSA 
ELA results which are used as one of the two major components for making ESOL exit decisions will 


not be available, and an alternative method is needed. This Research Brief describes the development 
of such a method and assesses its potential effect. The results show that the model can be used for 
making ESOL exit decisions in 2019-2020 and that these decisions would be conservative: it is unlikely 
that a student would be exited from ESOL prematurely by using the proposed model. 


Introduction 

State Board of Education Rule 6A-6.0903 specifies that to be eligible for an exit from the English for 
Speakers of Other Languages (ESOL) program, a student must complete and score “Proficient” on the 
State’s English Language Proficiency Assessment, ACCESS for ELLs. In addition, students in grades 3-9 must 
achieve a level 3 or higher on the English Language Arts (ELA) component of the Florida Standards 
Assessment (FSA or FSAA) while students in grades 10-12 must satisfy the reading requirements for high 
school graduation via achieving a sufficiently high score on FSA/FSAA or a concordance score. 


Due to the immediate unusual circumstances presented during the 2019-2020 school year related to the 
COVID-19, FSA and FSAA testing will not be administered to students and the ACCESS for ELLs? was only 
partially completed and scores may not be available in time to make ESOL exit decisions. Given that at 
least one of the two main components used for making ESOL exit decisions is not available, other avenues 
to make such decisions are needed. 


In this Research Brief, we propose a model that can be used in making ESOL exit decisions using the 


existing source of data: reading scores on the iReady assessment. To develop this model, we used the 
2018-2019 ACCESS results (n=61,647) and the 2018-2019 iReady AP2 (administered in December 2018) 


1 In the rest of the document we refer to this English Language Proficiency assessment as ACCESS. 
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reading results (n=41,253). The number of students with scores on both assessments varied from the low 
of 2,416 for grade 8 students to the high of 6,510 for grade K students for a total of 40,726 results. 


Preliminary Analyses 

To judge the strength of a linear relationship between the iReady and ACCESS results, the correlation 
coefficients between the iReady reading scale scores and the ACCESS for ELLs composite scale scores were 
found separately for each of the K-8 grade levels. The ACCESS composite scale score represents the 
weighted average of scores in all four domains of the ACCESS, in which the scale scores in Reading and 
Writing modalities are weighted each at 35% of the total, and the scale scores in Speaking and Listening 
modalities are weighted at 15% each. The cases with the “Rush flag” on the iReady file were excluded 
from the preliminary analyses and from the model development. (The “Rush flag” indicates that a student 
spent very little time answering the iReady questions.) 


In the second analysis, the correlation coefficients between the iReady reading scale scores and a 
dichotomous indicator variable signifying whether a student scored proficient on ACCESS were found for 
each grade level. Scoring proficient is defined as achieving a Reading proficiency level of at least four and 
the Composite proficiency level of at least four. 


The values of these point-biserial correlation coefficients are necessarily smaller than those found in the 
first analysis. In fact, the maximum possible value of a point-biserial correlation between a normally 
distributed variable and a dichotomous variable is approximately .80 (Gradstein, 1986). It is achieved 
when the proportion of “successes” on the dichotomous variable is .5. The farther the proportion of 
successes is from .5, the smaller is the maximum possible value of a point-biserial correlation coefficient. 
It is only .26 when the proportion of successes is .99. 


The results of the correlational analyses are given in the table below. 


Grade N Scores iReady vs. ACCESS iReady vs. Proficient on ACCESS 
K 4821 .64 44 
1 5770 74 50 
2 5632 78 .69 
3 5504 76 48 
4 4288 .82 54 
5 3899 82 55 
6 2539 .80 50 
7 2486 81 51 
8 2398 .80 52 


The results shown in the shaded column in the table above demonstrate that the relationship between 
the iReady reading scores and ACCESS composite scores were sufficiently strong for the development of 
a predictive model (Zhang, Patel, & Ewing, 2014). Typically, a correlation of approximately 0.4-0.5 is 
considered moderate, and a correlation of 0.6 or above is considered strong. The authors of the AP 
Potential study used 0.4 as the minimum value of the correlation for running prediction models. 


Model Development 

A binary logistic regression with the dichotomous indicator of proficiency on ACCESS as an outcome and 

a readings scale score on iReady as a predictor was fitted. The results indicated that the iReady reading 
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scores were a Statistically significant predictor of scoring proficient on ACCESS (p <.001). Then, the 
predicted probability of scoring proficient on ACCESS was calculated separately for each grade level. The 
minimum iReady reading scale scores corresponding to the probability of scoring proficient on ACCESS of 
at least 0.6 are shown in the table below?. These values were selected as cut scores for predicting students 
likely to score proficient on ACCESS. 


Grade | Scale Score 
0 426 
482 
505 
532 
529 
547 
585 
598 
608 
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Model Effectiveness 

After these cut scores were applied to all available AP2 2018-2019 iReady data, the prediction accuracy 
percentages were computed. These were found as the percentages that the total of true negative and 
true positive cases made of the total number of students with scores on both assessments per grade level. 
Here, true negative cases are those that were predicted to not score proficient and, in fact, did not score 
proficient on ACCESS, while true positive cases are those who were predicted to score proficient on 
ACCESS and did so. These accuracy percentages are contrasted with chance agreement percentages? 
(White, 2013) in the table below. 


Grade N Scores | Prediction Accuracy | Chance Agreement 
0 6557 88% 77% 
1 6621 89% 78% 
2 6205 82% 63% 
3 5808 78% 60% 
4 4436 76% 52% 


It can be seen that the prediction model accuracy percentages are substantially higher than chance 
agreement percentages in all grade levels. 


Impact Data 
To assess the model effect, we calculated the proportions of students predicted to score proficient on 
ACCESS based on the iReady data and proportions of such students who were in the ESOL level 4 at the 


* The predicted probability of 0.6 is often selected as a cut point in various prediction models, such as those used in 
the District’s Interim Assessment. In addition, this relatively high value is expected to result in a conservative 
prediction assuring that the students are not exited from the ESOL program prematurely. 
3 The chance agreement percentages were computed under the assumption that the percentages of students 
scoring proficient on ACCESS were known and were applied to classify students. For example, in grade K, 13% of 
students scored proficient on ACCESS. If one assigns 13% of students at random to the “proficient group”, and the 
remaining 87% to the “not proficient” group, then 77% of students would be expected to be classified correctly by 
chance. 
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end of 2018-2019. This was done because the District contemplates establishing a rule under which 
students who are in ESOL level 4 and are predicted to score proficient on ACCESS would be subject to 
exiting from the ESOL program. 


We compared those with observed percentages of students who scored proficient on ACCESS and 
proportions of students who exited the ESOL program at the end of the 2018-2019 school year. These 
percentages are shown in the table below. 


Predicted Proficient | Subject to ESOL Exit | Observed Proficient Observed Exited 
Grade Percentage Percentage Percentage ESOL Percentage 
0 3% 2% 13% 13% 
1 6% 3% 13% 13% 
2 16% 7% 26% 26% 
3 12% 4% 27% 19% 
4 30% 13% 40% 19% 
5 30% 18% 38% 14% 
6 10% 5% 23% 9% 
7 12% 6% 22% 9% 


The results in the table above demonstrate that the proportions of students predicted to score proficient 
on ACCESS were lower than the observed percentages. Similarly, the percentages of students who would 
be subject to exit from the ESOL program based on having been predicted to score proficient on ACCESS 
and their ESOL 4 status were lower than the percentages of students who exited the ESOL program in 
2018-2019. 


Discussion 

The results from the model presented above indicate that the developed model is conservative; it 
attempts to assure that no student will be exited from the ESOL program prematurely. The cut scores 
developed above can be applied to the 2019-2020 iReady AP2 data to make ESOL exit decisions under the 
current circumstances and in the absence of ACCESS and FSA ELA data. If and when 2019-2020 ACCESS 
results become available, these can be used to make ESOL exit decisions for students who didn’t qualify 
through this model. 
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